Method and apparatus for controlling contents in realtime fashion

ABSTRACT

Method for performing actual operation on a second device having a second screen or a touch panel from a first device having a first screen or a touch panel in real time are provided. First screen is configured to display mirrored content of second screen. Method includes detecting a first touch signal on first screen of first device, converting first touch signal into a first touch data associated with a first position information, and transmitting first touch data to second device so actual operation is performed on second device. For first and second devices having touch panels, method includes receiving a first touch data associated with a first position information of first touch panel, calculating a second position information of second touch panel based on first position information, and performing actual operation on second device based on second position information. Second device is coupled to first device by network communication.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation application of U.S. non-provisionalapplication Ser. No. 13/184584, filed on Jul. 18, 2011 and now pending,which in turn claims the benefit of U.S. provisional application Ser.No. 61/410,371, filed on Nov. 05, 2010, now abandoned, the contents ofwhich are incorporated herein by reference and made a part of thisdisclosure.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a method and an apparatus forcontrolling contents in realtime fashion, and more particularly, to amethod and an apparatus for controlling multimedia contents in realtimefashion by means of using wireless touch, motion sensing and batterycharging.

2. Description of the Prior Art

As the technology related to portable electronic device advances and theneed for portability increases in relation to the current market trend,traditional personal computers are transforming to lighter and morededicated devices provided for or targeted at different purposes anduser groups including such as gaming, telecommunication, web browsingand media streaming. Such shift of market and technology trends relatedto advanced needs of portability and interactions demanded by users ofsuch electronic devices are in fact facilitating not only theimprovement of traditional personal computers, laptops or smart-phonesto be more powerful and/or integrated with more features and functionsthan ever but also a new category or breed of portable electronicdevices with more dedicated functions for certain purposes and greaterportability at the same time.

In view of the merits of greater portability as well as interactionprovided by such newly evolved breed of electronic devices previouslymentioned, the inventor realizes that electronic designers andmanufacturers are introducing a “lighter” version of portable personalcomputers or laptop to the market while telecommunication devicedesigners and manufacturers are, in general, offering a “heavier”version of mobile phones such as smart-phones such that the new growingmarket and user demand of high portability as well as interactivefunctions may be fulfilled. Some known examples of such new breed ofdevices may include: iPad by Apple®, Playbook by RIM®, Cius by CISCO®,Galaxy Tab by Samsung®, Libretto by Toshiba®; and examples of knownsmart phones may be: iPhone by Apple®, Blackberry Bold by RIM®, Galaxy Sby Samsung®, HD7 by HTC™. In general, these portable devices share somecommon building blocks of traditional personal computers but withgreater portability in terms of their size, weight as well as withgreater interactive and integrated functions with more dedicatedapplications and telecommunication capabilities including via such asknown 3G, 4G or Wi-Fi™.

As such new breed of portable electronic devices are being introduced tothe market, the inventor also realizes that each one of such devices maybe operated under a specific platform or OS that may be unique from eachother, which may be good in providing varieties to the market in generalbut may too raise a preference issue to the user depending on his/herown demands in terms of such as function, design or performance ofdifferent operating systems and devices. Instead of choosing oneparticular type of portable electronic device miming a certain type ofoperating systems including such as Windows® by Microsoft®, iOS® byApple®, Android® or Linux® by Google®, Blackberry® OS by RIM®, theinventor realizes an user may advantageously be provided with anuniversal apparatus capable of communicating with the any example of newbreed of electronic devices running any certain type of operatingsystems and may be advantageously free from the preference selectionissue while still being able to utilize or interact with the any onesuch new breed of electronic devices running different type of operatingsystems.

However, there are at least two major concerns to be considered inproviding an electronic apparatus capable of advantageouslycommunicating and interacting with the newly introduced breed ofportable electronic devices, one is related to the various differentplatforms or operating systems being adapted by different electronicdevices and the other is to preserve the merits of great portability andbeing light in terms of operation and hardware structures or componentsof portable electronic devices demanded by the market trend. Therefore,there is a need to provide an electronic apparatus capable ofcommunicating with external device(s) of various types of operatingsystems while preserving a relatively “light” hardware structures toprovide a highly portable and interactive devices to users.

Furthermore, as previously mentioned that portable devices areintegrated with more functions to be easily accessed by and interactedwith users for more dedicated purposes or services including such asgaming, web browsing, teleconferencing and navigation. The inventorrealizes that in order to provide an electronic apparatus capable ofcommunicating with external devices of various of different operatingsystems offering more dedicated services or applications, the mostaccessible functions or main services of the communicated externaldevices shall be preferably preserved, controlled and/or interacted byusers while preserving a relatively less hardware structures to providea highly portable and interactive devices to users.

In view of the foregoing, it is desirable to provide an electronicapparatus capable of communicating with external device(s) of various ofdifferent types of operating systems such that applications and servicesof the connected/communicated external device of different types ofoperating systems in relation to such as interactive gaming,teleconferencing, web browsing, media playing, navigation, imagedisplaying or taking may all be preserved or advantageously madeavailable, and preferably with enhanced data transmission, whilerealizing or having a relatively “light” hardware structures with greatportability and interactive feasibility to users.

SUMMARY OF THE INVENTION

In one aspect, the present invention provides a method for performing anactual operation on a second device having a second screen from a firstdevice having a first screen in real time, in which the first device iscoupled to the second device via a network communication, the firstscreen is configured to display a mirrored content of the second screen,and the first screen includes touch panel capability.

In an embodiment, the method for performing the actual operation on thesecond device having the second screen from the first device having thefirst screen in real time includes the following steps: detecting afirst touch signal on the first screen of the first device; convertingthe first touch signal into a first touch data associated with a firstposition information with respect to a first coordinate system of thefirst screen; transmitting the first touch data to the second device viathe network communication, so that the actual operation is performed onthe second device. In this embodiment, the actual operation performed onthe second device can be overridden if a detecting event is detected onthe second screen of the second device. In this embodiment, the transmit(TX) side is also referred to as a second device, and the receive (RX)side is also referred to a first device.

In addition, optional steps of the embodiment include of the following:detecting a motion event on the first device and generating a sensordata, and sending the sensor data to the second device via the networkcommunication for enabling a corresponding operation on the seconddevice, in which the corresponding operation includes changingorientation of the content displayed on the second screen. In the methodof the present embodiment, one of a plurality of sensing ranges can beselected for detecting the motion event, and a threshold value can beprovided for detecting the motion event on the first device. The methodfurther includes charging capability to charge the first device with thesecond device or to charge the second device with the first device bydetecting a charging connection between the first device and the seconddevice, retrieving remaining battery amounts from the first device andthe second device respectively, and providing an option to charge thefirst device with the second device, or to charge the second device withthe first device.

In an another embodiment, a method for performing an actual operation ona second device having a second screen from a first device having afirst screen in real time further includes calculating a second positioninformation with respect to a second coordinate system of the secondscreen based on the first position information.

In yet another embodiment, the method for performing the actualoperation on the second device having the second screen from the firstdevice having the first screen in real time of the embodiment furtherincludes additional steps which include: receiving compressed multimediacontent compressed with a compression ratio from the second device viathe network communication, and decompressing and displaying themultimedia content on the first screen of the first device. Thecompression ratio is determined based on a bandwidth of the networkcommunication. In this embodiment, the decompressed multimedia contentcan be scaled based on a resolution of the first screen.

In another aspect, the present invention provides a method forperforming an actual operation on a second device having a second touchpanel from a first device having a first touch panel in real time, andthe first device is coupled to the second device via a networkcommunication.

In one embodiment, the method for performing the actual operation on thesecond device having the second touch panel from the first device havingthe first touch panel in real time includes steps as follow: receiving afirst touch data associated with a first position information withrespect to a first coordinate system of the first touch panel;calculating a second position information with respect to a secondcoordinate system of the second touch panel based on the first positioninformation; and performing the actual operation on the second devicebased on the second position information. In this embodiment, thetransmit (TX) side is also referred to as a second device, and thereceive (RX) side is also referred to a first device. In thisembodiment, the performing of the actual operation is overridden if adetecting event is detected on the second touch panel of the seconddevice. The method of this embodiment may further include steps forcompressing and decompressing of multimedia content from the seconddevice to the first device as follow: compressing a multimedia contentdisplayed on the second touch panel, and sending the compressedmultimedia content from the second device to the first device fordecompressing and displaying on the first touch panel via the networkcommunication. This embodiment may further include steps for performingoperations on the second device in response to sensor data indicating amotion event on the first device as follow: receiving a sensor data fromthe first device, the sensor data indicating a motion event on the firstdevice, and performing a corresponding operation on the second device inresponse to the sensor data. The corresponding operation may includechanging orientation of the multimedia content displayed on the seconddisplay.

In another aspect, the present invention provides a second deviceconfigured to be controlled by a first device with a first touch panel,and the first device is coupled to the second device by a networkcommunication.

In one embodiment, the second device includes a second touch panel, acompression unit for compressing multimedia content with a compressionratio, a transceiver for transmitting the compressed multimedia contentto the first device for decompressing and displaying on the first touchpanel and for receiving a touch data associated with a first positioninformation with respect to a first coordinate system of the first touchpanel, a coordinate calculating unit for calculating a second positioninformation with respect to a second coordinate system of the secondtouch panel, and a processor for performing an actual operation on thesecond device according to the touch data and the second positioninformation. In the second device, the actual operation can beoverridden if the second touch panel detects a detecting event or if thesecond device receives an incoming call.

The compression ratio is determined based on a bandwidth of the networkcommunication.

In the above embodiments, the transmit (TX) side is also referred to asa second device, and the receive (RX) side is also referred to a firstdevice; in addition, the first touch data is referred to the touch databeing formed after converting of the at least one touch signal intotouch data.

This summary is not intended to identify key features or essentialfeatures of the claimed subject matter, nor is it intended to be used tolimit the scope of the claimed subject matter.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments of the present invention will now be described, by way ofexample, with reference to the accompanying drawings, in which:

FIG. 1A is a schematic diagram showing wireless control from a RX sideover a TX side according to an embodiment of the present invention.

FIG. 1B and FIG. 1C are schematic diagrams showing touch controloperations of from the RX side over the TX side shown in FIG. 1Aaccording to an embodiment of the present invention.

FIG. 2 is a schematic diagram showing a compression ratio in relation toan available bandwidth between the RX side and the TX side according toan alteration of the present invention.

FIG. 3A and FIG. 3B are schematic diagrams showing an image orientationon the TX side in response to a motion detection on the RX side whereFIG. 3B is further provided with a switch element for adjusting motionsensitivity according to another alteration of the present invention.

FIG. 3C is a schematic diagram showing motion control operations of theRX side over the TX side corresponding to FIGS. 3A and 3B according toan embodiment of the present invention.

FIG. 3D and FIG. 3E are schematic diagrams showing adjustment of motionsensitivity on the RX side implemented by an application program on theTX side according to a further alteration of the present invention.

FIG. 3F and FIG. 3G are schematic diagrams showing adjustment ofmagnitude and/or threshold values of application programs on the TX sideafter motion data is received according to a further alteration of thepresent invention.

FIG. 4 is a flow diagram showing a process for controlling multimediacontents in realtime fashion according to an embodiment of the presentinvention.

FIG. 5 is a flow diagram showing a process for compressing anddecompressing multimedia contents in realtime fashion according to anembodiment of the present invention.

FIG. 6 is a flow diagram showing another process for compressing anddecompressing multimedia contents in realtime fashion according to anembodiment of the present invention.

FIG. 7A and FIG. 7B are schematic diagrams showing battery chargingbetween the TX side and the RX side according to a further alteration ofthe present invention.

FIG. 8 is a flow diagram of a battery charging process according to anembodiment of the present invention.

FIG. 9 is a flow diagram of a battery charging process according toanother embodiment of the present invention.

DETAILED DESCRIPTION

Refer to FIG. 1A, which is a schematic diagram of a RX side 10 (e.g.pad-like device having a touch-control display) and a TX side 12 (e.g.phone-like device having a computing power & operating system therein)according to an embodiment of the present invention. Advantageously, theRX side 10 is provided for controlling in realtime fashion multimediacontents that are originated from the TX side 12. The multimediacontents can include audio and video streams and be processed to presentat the TX side 12 (such as cellphone, smartphone, PDA, and GPSnavigation, or any other portable wireless device) having a phone-likescreen 1202 and the TX side 12 is coupled to the RX side 10 via anetwork communication NC, which is preferably a wireless communicationand a two-way communication. Noticeably, the multimedia contents mayinclude video and audio data, i.e. the video data can be displayed onthe screen 1202 while the audio data can be played via a speaker (notshown) in the TX side 12. Besides, the RX side 10 may be advantageouslyprovided in a CPU-less (without central processing unit) and/or GPU-less(without graphical processing unit) environment for the sake of lesspower consumption and/or simplified circuit complexity.

As shown in FIG. 1A, the TX side 12 may be a computing system or mobiledevice equipped with CPU and/or GPU computing power, such as a PC,laptop, tablet PC, mobile phone, smart-phone, and may include aprocessor (not shown) for executing an application program (not shown)of an operation system (OS) 1204 installed in a storage medium (notshown) and a transceiver 1206 coupled to the processor and capable oftransmitting and receiving the multimedia contents generated by theapplication program of OS 1204 executed by the processor. The OS 1204may be any one of existing or known OS, such as the abovementionedWindows®, iOS®, Android®, Linux®, Blackberry®, Symbian®, Palm®,installed on the TX side 12 such that an application program may beperformed or run thereon. The network communication NC between the RXside 10 and the TX side 12 may preferably be any one of the networkcommunication standards or profiles of: IEEE 802.11, IEEE 802.16,CDMA2000, WCDMA, TD-SCDMA, LTE, TDLTE, Bluetooth, Zigbee and combinationthereof, or wired communication such as ADSL network, Optical/Fibernetwork or Cable network.

The RX side 10 (refer back to FIG. 1A) includes a pad-like screen 1002(e.g. touch panel), a coordinate mapping unit 1004 and a transceiver1006. The screen 1002 is provided for detecting at least one touchsignal, and is preferably larger than the screen 1202. The coordinatemapping unit 1004 can be controlled and/or controllable by an operatingsystem (not shown). The coordinate mapping unit 1004 is provided forconverting the at least one touch signal (from the screen 1002) intotouch data associated with a first position information that indicates avirtual operation on the first screen 1002 corresponding to an actualoperation on the second screen 1202, wherein the first positioninformation is with respect to a first coordinate system of the firstscreen. For example, referred to FIG. 1B, the first position informationrepresents a current cursor position (right side of FIG. 1B) on thepad-like screen 1202 moved by the user's finger (not shown), and thusthe mirrored cursor on the phone-like screen 1202 will be moved to thecorresponding position (left side of FIG. 1B) such that the user can doany operation on the phone-like screen by touching the pad-like screen1202 while without touching the phone-like screen 1202. The transceiver1006 is provided for transmitting the touch data (e.g. the firstposition information indicative of the current position of cursor onscreen 1002) sent from the coordinate mapping unit 1004 to the TX side12 via the network communication NC. On the other hand, the TX side 12further includes a coordinate calculating unit 1208 for calculating asecond position information (e.g. new cursor position on phone-likescreen) with respect to a second coordinate system of the screen 1202based on the first position information. For example, when the user“points and clicks” the new object on the touch screen 1002 (RX), andthus the same new object on the touch screen 1202 (TX) will be “pointedand clicked” simultaneously because an absolute position (x′, y′) of thesame new object on the touch screen 1002 can be obtained as followingexpression:

-   -   the mapping absolute position (x′, y′) on the phone-like        screen=the current position (x, y)×mapping_ratio,        where    -   mapping_ratio along x-axis=x-axis_resolution of phone-like        screen/x-axis_resolution of pad-like screen;    -   mapping_ratio along y-axis=y-axis_resolution of phone-like        screen/y-axis_resolution of pad-like screen;    -   given the same origin for both screens 1202 and 1002.

Alternatively, the mapping movement (delta_x′, delta_y′) on thephone-like screen can be obtained as following expression:

-   -   mapping movement (delta_x′, delta_y′) on the phone-like        screen=current movement (delta_x, delta_v) on the pad-like        screen×mapping ratio,        where    -   delta_x=x1−x0; delta_y=y1−y0;    -   mapping_ratio along x-axis=x-axis_resolution of phone-like        screen/x-axis resolution of pad-like screen;    -   mapping_ratio along y-axis=y-axis_resolution of phone-like        screen/y-axis resolution of pad-like screen;        given the same origin for both screens 1202 and 1002,    -   where xo and y0 represents the old position on the pad-like        screen, and x1 and y1 represents the new position on the        pad-like screen while each of the mapping ratios has a value        less than one when the display size of the pad-like screen is        larger than that of the phone-like screen. Thus, the operating        system 1204 can be further provided for performing the actual        operation at the second screen based on the second position        information, such that the screen 1002 can display results from        the actual operation. As a result, the RX side 10 can control        multimedia contents played at TX side 12 in realtime fashion due        to up-scale and down-scale operations by the coordinate mapping        unit 1004 (RX side) and the coordinate calculating unit 1208 (TX        side).

Noticeably, the first position information may further include thecurrent position of the cursor on the pad-like screen 1002 of the RXside 10, and the resolution data of the RX side 10, e.g. 1920×1080, andthe second position information can be calculated so as to obtain themapping position of the cursor on the phone-like screen 1202, e.g.scaling down to a resolution 480×320. For example, the actual action canbe “play a video movie” when the mapping position of the cursor on thescreen 1202 is pointed on a “play” icon. As a result, the user can“touch” on the pad-like screen 1002 of the RX side 10 to implementoperations without access to the phone-like screen 1202 as if whathe/she usually does on the phone-like screen 1202 of the TX side 12. Onthe other hand, all the tasks requiring extensive computingpower/resources are handled by the TX side 12, and only the display taskis duplicated (mirrored) to the RX side 10 from the TX side 12 such thatthe multimedia contents can be presented in realtime fashion between thepad-like screen 1002 of the RX side and the phone-like screen 1202 ofthe TX side 12.

For example, please refer to FIG. 1B and FIG. 1C, which are schematicdiagrams of touch control operations of the RX side 10 and the TX side12 shown in FIG. 1A according to an embodiment of the present invention.As shown in FIG. 1B, a user can do the “open” operation to open a fileor folder on the larger screen 1002 of the RX side 10 (right side) andquasi-simultaneously the smaller screen 1202 of the TX side 12 (leftside) will mirror the same scene/image as that in the RX side 10, andthus the user can do the “double-click” operation to open some objectlike file/folder on the larger screen 1002 while utilize computing powerfrom the TX side 12, i.e. the operating system 1204, to run the archiveapplication. As shown in FIG. 1C, a user can do the “key-in” operationon the larger RX side 10 (right side) to type words and simultaneouslythe smaller screen 1202 of the TX side 12 (left side) will mirror thesame scene/image as that in the RX side 10, and thus the user can do the“key-in” operation on the larger screen 1002 while utilize computingpower from the TX side 12, i.e. the operating system 1204, to run thetext-editing application.

In detail, please continue to refer to FIG. 1A, the TX side 12 mayfurther include a compressing unit 1210, for compressing the multimediacontents with a compression ratio, e.g. compressing RGB data into MJPEGdata, which requires less time for compression and thus facilitatesoperating in realtime fashion, and the transceiver 1206 can be furtherutilized for transmitting the compressed multimedia contents to thescreen 1002 for displaying. Specifically, the transceiver 1006 receivesthe compressed multimedia contents from the transceiver 1206 via thenetwork communication NC, and then compressed multimedia contents aredecompressed by a decompressing unit 1010 with the compression ratio,such that the video data of the multimedia contents can be displayed onthe screen 1002 while the audio data of the multimedia contents can beplayed by a speaker 1008. As a result, the multimedia contents can bedisplayed and controlled in realtime fashion between the screen 1002 ofthe RX side 10 and the screen 1202 of the TX side 12.

Moreover, please refer to FIG. 2, which is a schematic diagram of the RXside 10 and the TX side 12 according to an alteration of the presentinvention. In the embodiment shown in FIG. 2, the transceiver 1206 canfurther detect a bandwidth of the network communication NC, and the TXside 12 further includes a dynamic compression unit 1212, fordetermining the compression ratio of the multimedia content based on thebandwidth of the network communication NC, such that the compressingunit 1210 can compress the multimedia contents at the compression ratio,which is determined by the bandwidth of the network communication NC.Correspondingly, the transceiver 1006 also can detect the bandwidth ofthe network communication NC, and then the decompressing unit 1010decompresses the compressed multimedia contents by the compression ratioprovided by a bandwidth calculating unit 1012. As a result, themultimedia contents can be compressed with a compression ratio accordingto a bandwidth of the network communication NC, such that the multimediacontents can be smoothly displayed in realtime fashion between thescreen 1002 of the RX side 10 and the screen 1202 of the TX side 12 byfacilitating the realtime codec function (e.g. the video compressionratio is determined by the actual available bandwidth). Besides, thetransceiver 1006 (RX side) also can directly receive the compressionratio from the dynamic compression unit 1212 (TX side) rather than thebandwidth calculating unit 1012, and then to pass the compression ratiointo the decompressing unit 1010 so that the bandwidth calculating unit1012 can be removed as well.

Besides, the present invention discloses another method for controllingin realtime fashion multimedia contents between a RX side having a firsttouch panel and a TX side having a second touch panel via a networkcommunication. The multimedia contents comprise at least a screen framedata (not shown) where the screen frame data includes the screenresolution data, image data displayed on the screen and audio data. Themethod comprising the steps of:

-   -   Step 1: detecting at least one touch signal from the second        touch panel 1202, for example, the user may touch the        touch-sensitive phone-like screen 1202 to do something, and the        “touch” can be detected and a touch signal will be sent to OS        for particular operation;    -   Step 2: in response to detecting the touch signal from the        second touch panel, compressing the screen frame data on the TX        side and transmitting to the RX side so that the RX side        decompresses the compressed screen frame data so as to obtain a        TX's resolution data, where the TX's resolution data is provided        together with a RX's resolution data to scale the screen frame        data so as to display the screen frame data to fit with the RX's        touch panel;    -   Step 3: detecting at least one touch signal from the first touch        panel;    -   Step 4: in response to detecting the touch signal from the first        touch panel, the further steps including:    -   Step 4-1: converting the touch signal on the RX side into touch        data associated with a first position information defining a        virtual operation on the first screen corresponding to an actual        operation on the second screen, the first position information        being with respect to a first coordinate system of the first        screen;    -   Step 4-2: transmitting the touch data from the RX side to the TX        side via the network communication, and calculating on the TX        side a second position information with respect to a second        coordinate system of the second screen based on the first        position information, and    -   Step 4-3: performing the actual operation at the second screen        based on the second position information; and    -   Step 5: implementing an override mechanism to assign the TX side        a higher priority overriding the RX side when the two detecting        events (respectively from the TX and RX sides) occur        simultaneously, for example, the phone-like screen 1202 will        override the pad-like screen 1002 to take control when the user        receives an incoming call from the phone-like screen 1202, and        the override mechanism is implemented by a software program        installed at the TX side. That means the TX side can dominant        the RX side to facilitate the communication since the user has        paid attention on doing the task at the TX side rather the RX        side (e.g. pick-up the incoming call).

Furthermore, please refer to FIGS. 3A-3B, which are schematic diagramsof the TX side 12 and the RX side 10 according to another alteration ofthe present invention. In one embodiment shown in FIG. 3B, the RX side10 may further include a motion sensor 1014 (including at least one ofan accelerometer, a gyroscope and a magnetic compass) for detecting atleast one motion event. The receiver 1006 transmits the sensor data (RX)from the motion sensor 1014 after the motion sensor 1014 has detectedthe motion event on the RX side, and then the sensor data (RX) is sentto the TX side 12 via the network communication NC. A sensor decisionunit (TX) may disable sensor data (TX) generated from a motion sensor1216 on the TX side 12 only if the detecting exists (i.e. the sensordata (RX) from the RX side has being received on the TX side), and thensend the sensor data (RX) to a motion process unit 1214. The motionprocess unit 1214 can convert the sensor data (RX) into motion data(indicating a translation representative of linear acceleration inthree-axis directions, and/or an angular rotation representative of Yaw,Roll and Pitch in three-axis directions) on the TX side 12, so as topresent the multimedia contents on the screen 1202 and subsequently onthe screen 1002 in a same particular orientation (e.g. portraitorientation or landscape orientation). The particular orientation isdetermined by the operating system 1204 according to the motion data(RX) such that the image at the screen 1202 will be presented at thelandscape or portrait orientation. As a result, the multimedia contentscan be displayed and controlled in realtime fashion between the screen1002 of the RX side 10 and the screen 1202 of the TX side 12 throughmotion operations.

For example, please refer to FIG. 3C, which is a schematic diagrams ofmotion control operations between the RX side 10 and the TX side 12shown in FIGS. 3A-3B according to one embodiment of the presentinvention. As shown in the right side of FIG. 3C, a user can do the“game play” driving motion, e.g. moving the image (RX) of the largerpad-like screen 1002 from Landscape to Portrait, and thus the mirroredimage (TX) on the smaller phone-like screen 1002 (shown on left side ofFIG. 3C) is quasi-simultaneously adjusted from Landscape to Portrait onthe TX side 12 performed by the operating system 1204 based on themotion data converted from the sensor data (RX) generated by the motionsensor 1014 (RX). Thus, the user can play the game on the pad-likescreen 1002 while utilize computing power from the TX side 12, i.e. theoperating system 1204, to run the game application actually on thephone-like screen 1202.

Noticeably, the motion sensor 1014 may have a plurality of sensingranges, i.e. different sensing sensitivities, or a particular sensingrange for generating the sensor data. When the motion sensor 1014 has aplurality of sensing ranges, one of the multiple sensing ranges can beselected by a control code determined by a switch signal so as to enablea motion chip to generate one motion data based on the selected sensingrange determined by the control code.

In a first embodiment, the motion sensor 1014 has a plurality of sensingranges. As shown in FIG. 3A-3B, the RX side 10 detects whether a switchsignal is invoked by a switch element 1016 operative to a user, and theswitch element 1016 may issue an adjustment signal to a motion chip(i.e. motion sensor 1014) after the switch signal was invoked (e.g. userturns the switch element 1016 on). Then, the motion sensor 1014 selectsone of the plurality of sensing ranges of the motion chip based on theadjustment signal, and generates the motion data by the motion chipbased on the selected sensing range so as to enable an interactionbetween the RX side 10 and the TX side 12. For example, the motion chiphas a sensing range of ±1 G at its “off” (original) state and ±2 G at“on” state. When the switch element 1016 is turned to “on” from “off” bythe user, the motion chip may have a new sensing range of ±2 G, where Gis defined as 9.81 m/s² for gravitational acceleration.

In a second embodiment, the motion sensor 1014 has a plurality ofsensing ranges. As shown in FIG. 3D-3E, the TX side 12 detects whether aswitch signal is invoked by an application program 1220 (e.g. gamesoftware/program), and an adjustment signal is generated in response tothe invoked switch signal. Then, the motion sensor 1014 selects one ofthe plurality of sensing ranges of the motion chip of the motion sensor1014 based on the adjustment signal, and generates the motion data bythe motion chip based on the selected sensing range so as to enable aninteraction between the RX side 10 and the TX side 12. For example, thegame software may provide an input user interface to allow the user tochoose one of three options for selecting particular sensing range amongthe plurality of sensing ranges where Option1 stands for ±1 G; Option2for ±2 G; and Option3 for ±3 G, respectively. A particular switch signalwill be sent out to the RX side after the selected Option2 is chosen bythe user, for instance. The motion sensor 1014 (RX) is requested tochange its sensing range to ±2 G directly by the switch signal, or by anadjusted signal in response to the switch signal.

In a third embodiment, the motion sensor 1014 has a particular sensingrange for generating the sensor data. As shown in FIG. 3F-3G, the TXside 12 detects whether a switch signal is invoked by an applicationprogram 1222 of the operating system 1204, and an adjustment signal issent in response to the invoked switch signal. Then, the TX side 12 mayprovide a user interface to allow the user to adjust a threshold valueof the application program 1222 (e.g. game software/program) based oneither the adjustment signal or the invoked switch signal such that thethreshold value of the application program 1222 can be increased at aratio corresponding to the adjustment signal or the invoked switchsignal. Subsequently, the motion data may be generated based on sensordata that is sent from the motion sensor 1014 (on the TX side) triggeredby the switch signal, so as to enable an interaction between the RX side10 and the TX side 12.

In a fourth embodiment, the motion sensor 1014 has a particular sensingrange for generating the sensor data. As shown in FIG. 3D, the TX side12 detects whether a switch signal is invoked by the application program1222 of the operating system 1204, and issues an adjustment signal inresponse to the invoked switch signal. Then, the TX side 12 may providea user interface to allow the user to adjust a magnitude of the motiondata by means of increasing the magnitude's value at a ratio based oneither the adjustment signal or the switch signal, so as to enable aninteraction between the RX side 10 and the TX side 12. Noticeably, inthe above third and fourth embodiment, the TX side 12 can be configuredto adjust the threshold value of the application program 1222 and themagnitude of the motion data, respectively, but the TX side 12 can alsosimultaneously adjust the threshold value of the application program1222 and the matmitude of the motion data, while retaining both merits.

Noticeably the spirit of the present invention is to utilize a RX sideto control multimedia contents played at transmitting entities ofdifferent types of operating systems in realtime fashion, wherein the RXside only transmits touch data and sensor data for a TX side to performactual operation, and displays the multimedia contents of the TX side.Those skilled in the art should make modifications or alterationsaccordingly. For example, the RX side 10 can be foldable, portable, orany other types to meet each user's need(s).

Besides, the network communication NC may preferably include a Wi-Ficommunication and a Bluetooth (BT) communication, and since the touchdata and sensor data have less data volume while the multimedia contentshave greater data volume, the touch data and the motion date arepreferably transmitted to the TX side 12 via the BT communication, whilethe multimedia contents are preferably transmitted to the RX side 10 viathe Wi-Fi communication. It can be understood that the transceiver 1006and 1206 of the present invention may be a wireless module adaptingother communication standards or profiles including such as IEEE 802.16(WiMAX), CDMA2000, WCDMA, TD-SCDMA, LTE, TDLTE, Bluetooth-profile andcombination thereof or a wired module.

Furthermore, the video data of the multimedia contents decompressed andoutputted by the decompressing unit 1010 may preferably from a MJPEGformat to a RGB format and transmitted or outputted to the screen 1002in a LVDS format via a display interface converter such as a RGB-to-LVDSconverter. It can be understood that other video stream formats such asITDMI and related interfaces may also be possible. On the other hand,the audio data of the multimedia contents processed and outputted maypreferably be transmitted via an audio interface such as an I²Sinterface or standard and further to the speaker 1008.

Moreover, it is preferable that the multimedia contents transmitted andprocessed in video streaming may be made “real-time” or withoutsignificant delays with respect to time for the sake of low latency. Inother words, in a preferred explanatory example of the present inventionrecited herein, the multimedia contents may be communicated between theRX side 10 and the TX side 12 and processed to output “real-time” videoand audio streams to the screen 1002 and/or the speaker 1008 of the RXside 10 of the present invention. Preferably, the multimedia contentsmay be processed in video streaming (i.e. encoding and decoding) andcommunicated there between as data packets and the duration or delay oftime involved in a “real-time” processing and transmission may be lessthan 100 ms for the sake of low latency; in one example, it maypreferably be less than 10 ms, and in another example, it mayapproximately be 20 ms.

Operations of the RX side 10 and the TX side 12 shown in FIG IA can besummarized into a controlling process 40 shown in FIG. 4. Thecontrolling process 40 includes the following steps:

-   -   Step 400: Detect at least one touch signal via the screen 1002,        and convert the at least one touch signal into touch data        associated with a first position information defining a virtual        operation on the screen 1002 corresponding to an actual        operation on the screen 1202, the first position information        being with respect to a first coordinate system of the screen        1002.    -   Step 402: Transmit the touch data to the TX side 12 via the        network communication NC, and calculate a second position        information with respect to a second coordinate system of the        screen 1202 based on the first position information.    -   Step 404: Perform the actual operation at the screen 1202 based        on the second position information, and display results from the        actual operation onto the screen 1002 so as to control the        multimedia contents in realtime fashion between the screen 1002        of the RX side 10 and the screen 1202 of the TX side 12.    -   Step 406: End.

Operations of the RX side 10 shown in FIG. 1A can be summarized into areceiving process 50 shown in FIG. 5. The receiving process 50 includesthe following steps:

-   -   Step 500: Receive compressed multimedia contents from the TX        side 12.    -   Step 502: Convert the multimedia contents for decompression.    -   Step 504: Scale the decompressed data based on a resolution of        the RX side 10.    -   Step 506: Output displayable multimedia contents to the RX side        10.    -   Step 508: Detect whether there is touch signal from the RX side        10. If yes, go to step 510; otherwise go to step 500.    -   Step 510: Output touch data to the TX side 12. Go to step 500.

Operations of the RX side 10 and the TX side 12 shown in FIG. 2A can besummarized into a receiving and transmitting process 60 shown in FIG. 6.The receiving and transmitting process 60 includes the following steps:

-   -   Step 600: Receive compressed multimedia contents from the TX        side 12.    -   Step 602: Determine a compression ratio based on a bandwidth of        the network communication NC.    -   Step 604: Convert the multimedia contents based on the        compression ratio.    -   Step 606: Decompress the compressed data, and scale thereof if        needed.    -   Step 608: Output displayable multimedia contents to the RX side        10.    -   Step 610: Detect whether there is touch signal from the RX side        10. If yes, go to step 510; otherwise go to step 600.    -   Step 612: Output touch data to obtain position data.    -   Step 614: Perform actions based on the position data at the TX        side 12. Go to step 600.

Detail descriptions of the controlling process 40, the receivingprocesses 50 and the receiving and transmitting process 60 can bederived by referring to the above description, and are not narratedhereinafter.

On the other hand, please refer to FIG. 7A-7B, which is a schematicdiagram of the RX side 10 according to another alteration of the presentinvention. As shown in FIG. 7A-7B, the TX side 12 further comprises acharging application 1224, charging detector 1226 and battery cells1228, and the RX side 10 further comprises a charging controller 1018and battery cells 1020.

In one embodiment, system default charging operations of the TX side 12and the RX side 10 shown in FIG. 7A-7B can be summarized into chargingprocess 80 as shown in FIG. 8. The charging process 80 includes thefollowing steps:

-   -   Step 800: Detect an event of a charging connection between the        TX side 12 and the RX side 10. The charging connection is        advantageously provided for conforming to the USB standard. It        is advantageously applicable to the two-way charging process.    -   Step 802: Retrieve remaining battery amounts (Rt and Rr)        respectively from battery cells 1228 on the TX side 12 and        battery cells 1020 on the RX side 10.    -   Step 804: Compare the two remaining battery amounts (Rt and Rr)        respectively with two predetermined battery capacity percentages        (Tt and Tr) by a power management mechanism that is executed by        either the charging application 1224 in the operating system or        the charging controller 1018. If each of the remaining battery        amounts is below the predetermined battery capacity percentage        (i.e. Rt<Tt and Rr<Tr), go to step 806; if the remaining battery        amount of the RX side 10 is higher than its predetermined        battery capacity percentage while the remaining battery amount        of the RX side 10 is higher than that of the TX side 12 (i.e.        Rr>Tr and Rr>Rt), go to step 810; if the remaining battery        amount of the TX side 12 is higher than its predetermined        battery capacity percentage while the remaining battery amount        of the TX side 12 is higher than that of the RX side 10 (i.e.        Rt>Tt and Rt>Rr), go to step 814.    -   Step 806: Send a first message of recharging information shown        as “Please Use Power Adaptor to charge Pad and Cellphone”.    -   Step 808: Charging operation 1: Use Power Adaptor to charge Pad        and Cellphone.    -   Step 810: Send a second message of recharging information shown        as “Please Use Pad to charge Cellphone”.    -   Step 812: Charging operation 2: Use Pad to charge Cellphone.    -   Step 814: Send a third message of recharging information “Please        Use Cellphone to charge Pad”.    -   Step 816: Charging operation 3: Use Cellphone to charge Pad.

In another embodiment, user selection charging operations of the TX side12 and the RX side 10 shown in FIG. 7A-7B can be summarized intocharging process 90 as shown in FIG. 9. The charging process 90 includesthe following steps:

-   -   Step 900: Detect an event of a charging connection between the        TX side 12 and the RX side 10. The charging connection is        advantageously provided for conforming to the USB standard. It        is also applicable to the two-way charging process.    -   Step 902: Retrieve remaining battery amounts (Rt and Rr)        respectively from battery cells 1228 on the TX side 12 and        battery cells 1020 on the RX side 10.    -   Step 904: Present a user interface to allow a user's selection        of battery charging.    -   Step 905: If a user chooses option 1 for charging Cellphone, go        to step 906; if the user chooses option 2 for charging Pad, go        to step 908.    -   Step 906: Use Pad to charge Cellphone.    -   Step 908: Use Cellphone to charge Pad.    -   Step 910: Compare the two remaining battery amounts with two        predetermined battery capacity percentages (Tt and Tr) by a        power management mechanism that is executed by either the        charging application 1224 in the operating system or the        charging controller 1018. If each of the remaining battery        amounts is below the predetermined battery capacity percentage        (i.e. Rt<Tt and Rr<Tr), go to step 912; if the remaining battery        amount of the RX side 10 is higher than its predetermined        battery capacity percentage, and simultaneously the remaining        battery amount of the RX side 10 is higher than that of the TX        side 12 (i.e. Rr>Tr and Rr>Rt), go to step 916; if the remaining        battery amount of the TX side 12 is higher than its        predetermined battery capacity percentage, and simultaneously        the remaining battery amount of the TX side 12 is higher than        that of the RX side 10 (i.e. Rt>Tt and Rt>Rr), go to step 920.    -   Step 912: Send a first message of recharging information shown        as “Please Use Power Adaptor to charge Pad and Cellphone”.    -   Step 914: Charging operation 1: Use Power Adaptor to charge Pad        and Cellphone. Come to END.    -   Step 916: Send a second message of recharging information shown        as “Please Use Pad to charge Cellphone”.    -   Step 918: Charging operation 2: Use Pad to charge Cellphone.        Come to END.    -   Step 920: Send a third message of recharging information “Please        Use Cellphone to charge Pad”.    -   Step 922: Charging operation 3: Use Cellphone to charge Pad.        Come to END.

The event of a charging connection can be detected either by a chargingdetector 1226 on the TX side 12 or by a charging controller 1018 on theRX side 10. The charging controller 1018 can be implemented by a powermanagement chip. The charging detector 1226 can receive the remainingbattery capacity of the battery cells 1228 on the TX side 12; so as tosend the battery capacity information to the user interface by means ofthe charging application 1224 managed by OS 1204, and thus users canview the icon that indicates the actual measurement of the TX's batterycapacity. The charging controller 1218 on the RX side 10 can receive theactual measurement of the RX's battery capacity, and sends themeasurement to the charging detector 1226 on the TX side 12, so as tofacilitate the charging detector 1226 to determine what condition can bemeet to implement the appropriate charging operation.

For example, when each of the remaining battery amounts is below thepredetermined battery capacity percentage (i.e. Rt<Tt and Rr<Tr), apower adaptor is used to charge both the RX side 10 and the TX side 12;when the remaining battery amount of the RX side 10 is higher than itspredetermined battery capacity percentage, and simultaneously theremaining battery amount of the RX side 10 is higher than that of the TXside 12 (i.e. Rr>Tr and Rr>Rt), the RX side 10 is used to charge the TXside 12; when the remaining battery amount of the TX side 12 is higherthan its predetermined battery capacity percentage, and simultaneouslythe remaining battery amount of the TX side 12 is higher than that ofthe RX side 10 (i.e. Rt>Tt and Rt>Rr), the TX side 12 is used to chargethe RX side 10. It is noted that the RX side may be compulsorilyswitched to the power saving mode by the override mechanism when theuser receives an incoming call from the TX side, that is, the usercurrently uses the TX side rather the RX side to either answer or dialthe call. Therefore, it makes sense for the user to temporarily keep theRX side idle while he or she is busy for phone call.

In the prior art, a universal apparatus capable of communicating withthe any example of new breed of electronic devices running any certaintype of operating systems is lack. In comparison, the present inventionutilizes a RX side to control multimedia contents played at transmittingentities of different types of operating systems in realtime fashion,wherein the RX side only transmits touch data and sensor data for a TXside to perform actual operation, and displays the multimedia contentsof the TX side. Furthermore, the present invention can compressmultimedia contents a bandwidth of network communication, such that themultimedia contents can be smoothly displayed in realtime fashionbetween the screen 1002 of the RX side 10 and the screen 1202 of the TXside 12. Besides, the TX side 12 can also be configured as a serverhaving encoding hardware and the RX side 10 as a client having decodinghardware adapted for cloud computing environment running video streamingfor the purpose of low latency requirement. Those skilled in the artwill readily observe that numerous modifications and alterations of thedevice and method may be made while retaining the teachings of theinvention. Accordingly, the above disclosure should be construed aslimited only by the metes and bounds of the appended claims.

What is claimed is:
 1. A method for performing an actual operation on asecond device having a second screen from a first device having a firstscreen in real time, the first device is coupled to the second devicevia a network communication, the first screen is configured to display amirrored content of the second screen, the first screen includes touchpanel capability, the method comprises: detecting a first touch signalon the first screen of the first device; converting the first touchsignal into a first touch data associated with a first positioninformation with respect to a first coordinate system of the firstscreen; and transmitting the first touch data to the second device viathe network communication, so that the actual operation is performed onthe second device.
 2. The method of claim 1, further comprising:calculating a second position information with respect to a secondcoordinate system of the second screen based on the first positioninformation.
 3. The method of claim 1, further comprising: receivingcompressed multimedia content compressed with a compression ratio fromthe second device via the network communication; and decompressing anddisplaying the multimedia content on the first screen of the firstdevice.
 4. The method of claim 3, wherein the compression ratio isdetermined based on a bandwidth of the network communication.
 5. Themethod of claim 3, further comprising: scaling the decompressedmultimedia content based on a resolution of the first screen.
 6. Themethod of claim 1, wherein the actual operation performed on the seconddevice is overridden if a detecting event is detected on the secondscreen of the second device.
 7. The method of claim 1, furthercomprising: detecting a motion event on the first device and generatinga sensor data; and sending the sensor data to the second device via thenetwork communication for enabling a corresponding operation on thesecond device.
 8. The method of claim 7, wherein the correspondingoperation includes changing orientation of the content displayed on thesecond screen.
 9. The method of claim 7, further comprising: selectingone of a plurality of sensing ranges for detecting the motion event. 10.The method of claim 7, further comprising: providing a threshold valuefor detecting the motion event on the first device.
 11. The method ofclaim 1, further comprising: detecting a charging connection between thefirst device and the second device; retrieving remaining battery amountsfrom the first device and the second device respectively; providing anoption to charge the first device with the second device, or to chargethe second device with the first device.
 12. A method for performing anactual operation on a second device having a second touch panel from afirst device having a first touch panel in real time, the first deviceis coupled to the second device via a network communication, the methodcomprises: receiving a first touch data associated with a first positioninformation with respect to a first coordinate system of the first touchpanel; calculating a second position information with respect to asecond coordinate system of the second touch panel based on the firstposition information; and performing the actual operation on the seconddevice based on the second position information.
 13. The method of claim12, further comprising: compressing a multimedia content displayed onthe second touch panel; and sending the compressed multimedia contentfrom the second device to the first device for decompressing anddisplaying on the first touch panel via the network communication. 14.The method of claim 12, wherein the performing of the actual operationis overridden if a detecting event is detected on the second touch panelof the second device.
 15. The method of claim 12, further comprising:receiving a sensor data from the first device, the sensor dataindicating a motion event on the first device; and performing acorresponding operation on the second device in response to the sensordata.
 16. The method of claim 15, wherein the corresponding operationincludes changing orientation of the multimedia content displayed on thesecond display.
 17. A second device configured to be controlled by afirst device with a first touch panel, the first device is coupled tothe second device by a network communication, the second devicecomprising: a second touch panel; a compression unit for compressingmultimedia content with a compression ratio; a transceiver fortransmitting the compressed multimedia content to the first device fordecompressing and displaying on the first touch panel, and for receivinga first touch data associated with a first position information withrespect to a first coordinate system of the first touch panel; acoordinate calculating unit for calculating a second positioninformation with respect to a second coordinate system of the secondtouch panel; and a processor for performing an actual operation on thesecond device according to the first touch data and the second positioninformation.
 18. The second device of claim 17, wherein the actualoperation is overridden if the second touch panel detects a detectingevent.
 19. The second device of claim 17, wherein the actual operationis overridden if the second device receives an incoming call.
 20. Thesecond device of claim 17, wherein the compression ratio is determinedbased on a bandwidth of the network communication.